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ABSTRACT 

Science teachers tend to focus on schemas rather than 
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structure is the recognition that the logical statement implication 
can only be falsified. After instruction with a modified version of 
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elementary teachers were significantly more eUale to interpret logical 
statements, and falsifiers were more successful than verifiers. This 
study demonstrates that appropriate instruction can lead preservice 
teachers to understand the process of hypotl^esis formation and 
falsification that is the procedure of theory testing in science. (19 
references) (Author) 
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ABSTRACT 



Science teachers tend to focus on schemes rather than 
underlying structures with greater explanatory power. One such 
structure is the recognition that the logical statement of 
implication can only be falsified. 

After instruction with a modified version of the four-card 
hypothesis testing task, pre-service elementary teachers were 
significantly more able to interpret logical statements, and 
falsifiers were more successful than verifiers. 

This study demonstrates that appropriate instruction can 
lead pre-service teachers to understand the process of hypothesis 
formation and falsification that is the procedure of theory 
testing in science. 



HYPOTHESIS TESTING 

Teaching a Hypothaslt Ttstlng Strategy 
To Prospaotiva Taach«rs 

Introduction 

Formal operational thought Is a psychological process that 
mirrors the nature of scientific Inquiry so closely that It Is 
little surprise that It should be heavily emphasized by science 
educators. This Is particularly true of the schemes, such as 
controlling variables or conservation principles, that provided 
the context within which Plaget was able to examine thought 
processes. However, Kuhn, Amsel, and O'Loughlln (1988) have been 
critical of a tendency to focus on schemes, rather than to 
emphasize underlying structures with greater explanatory power. 

Neo-Plagetlanlsm takes at least two forms. One attempts to 
Interpret logical thought as a consequence of a larger 
explanatory system, such as, perhaps. Information processing 
theory. The other seelcs to demonstrate that elements of other 
psychologies are entailed by Plagetlan theory. Claims of the 
latter sort have been made, for Instance, with regard to memory, 
cognitive style, and spatial ability. 

A candidate for consideration as a component of fon.al 
reasoning Is the ability to test hypotheses, and parti /lularly 
those that appear In the form of Implication statements such as 
"If p then q** or "all p's are q.** Using a particularly unusual 
example, Hempel (1970) addressed the proposition that "all 
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HYPOTHESIS TESTING 

mermaids ar« green", and showed that the truth of this hypothesis 
can only be tested by a search for mermaids that are not green. 
Because this Insight underlies the type of hypothetlco-deductlve 
thinking that Is characteristic of formal operational reasoning, 
(dell-Aqulla, Gennaro & Plcclarelll, 1985; Lavson, 1987), It Is 
especially Important that It be mastered by both students and 
teachers* 

Moshman (1979) Identified three necessary components of 
successful hypothesis testing: a) a correct analysis of the 
logical relationships presented by an Implication statement, b) a 
realization that Implications are not conclusively supported by 
verifying data, and c) an understanding that Implications can 
only be falsified. Hempel (1970) suggests that our difficulty In 
understanding that large numbers of green mermaids could not 
prove the hypothesis true Is the result of what he calls a 
psychological Illusion. Moshman refers. Instead, to a 
*nonverl float Ion Insight* and a 'falsification strategy* as 
requirements for such understanding. 

Hypothesis testing ability Is often assessed with the four- 
card task created by Wason and Johnson-Laird (1972). Subjects 
are presented with four cards showing *A*, *D*, *4*, *7*, and 
told that every card has a letter on one side and a number on the 
other. They are then asked which cards they would turn over to 
test the rule that ^It a card has a vowel on one side, then It 
has an even number on the other side.** 
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Becaus* th« original version of th« four card task was quits 
abstract and difficult, Bady (1979) crsatsd an altsrnativs 
version with plausibls scisntific contsnt. In this task, 
subjects are asked to consider the hypothesis that "all turtles 
with diamonds on their back have green bottoas". Bady and 
Enyeart (1978) also conceived this as an instructional tool to 
teach students this difficult concept in logic. In both 
instances, successful coapletion of the task or activity requires 
the insight that hypotheses cannot be conclusively proven, and 
the ability to use a falsification strategy. Since the 
hypothesis can only be falsified by diaaond-backed turtles with 
bottoas that are not green, the optiaal strategy is to search for 
turtles with diaaonds on their back to prove that they are not 
green or turtles that are not green to prove that they have 
diaaonds on their back. The conclusion that turtles with 
diaaonds and green prove nothing is very difficult for aany 
people to accept, as is the insight that hypothetical stateaents 
of this sort cannot be proven, but only disproven. 

f)ackground 

One of the principal objectives of science education is to 
improve the reasoning abilities of students. In conseqpience of 
this, aost teacher preparation prograas assuae that pre-service 
teachers will theaselves acquire such skills. Indeed, it is 
difficult to imagine a classrooa where children becoae adept at 
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critical thinking abilities that tha teacher has net already 
mastared. 

A generalized ability that appears to underlie scientific 
reasoning is logical thought as described by Piaget (Inhelder & 
Piaget, 1958). Although Piaget 's proposal was that formal 
operational reasoning ability would be arrived at by most 
individuals by the age of fifteen, more recent studies, and 
especially those in the United States, have indicated that this 
may be an unduly optimistic estimate. 

Studies with adolescent and college-age students indicate 
that only about one-third of that population are able to use 
formal operations (Kolodiy, 1975? Griffiths 1976). From among 
the remainder, about half are concrete operational and half are 
transitional. A study of 265 subjects in the range of age from 
ten to fifty years revealed that this frequency of formal 
operational thought does not change substantially with adulthood 
(Kuhn, Langer, Kohlberg & Haan, 1977). In a review of twenty- 
five studies of adolescent and adult subjects. King reported 
success rates of between 40 and 70 percent, and concluded that "a 
sizable proportion of the normal adult population does not reason 
at formal levels when tested on formal operations tasks (1986, 
pg. 15)." 

Teachers are rarely examined as a separate population. 
However, Reyes (1987) used Hans Purth's An Invntorv Qt PjaggtLa 
Developmental Tasks (IPDT) to assess the cognitive development of 
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pro8p«ctlv« •lementary and sacondary taachars. Almost half (45%) 
of the elenantary aducatlon majors vara avaluatad as concrata, 
and only 12% wara formal. Sacondary aducatlon studants vara 
somawhat mora succassful, yitt 21% formal and 28% concrata. Tha 
succass rata of a group of non-aducatlon studants anrollad in a 
requlrad spaach communications coursa was almost idantical to 
that of tha prospactiva alamantary school taachars. Thus it 
would appear that tha most optimistic astimata is that 
prospactiva taachars ara as advanced in tarms of their cognitive 
development as most adolescents or adults. 

Moshman and Thompson (1981) cdte a number of studies 
indicating that most people are inclined to test hypothetical 
statements by seeking information that would verify them, but 
that the falsification strategy becomes more common in the years 
between the seventh grade and college. For example, dell'Aquila, 
Gannaro and Picciarelli (1985) found that the frequency of 
falsification responses rose from 2.7 to 6.4 percent between 
grades 6 and 12, while the percentage of verification responses 
dropped from 19.4 to 6.4. Using an exceptionally stringent set 
of criteria, including a correct logical interpretation, 
identification of those cases that disconfirmed the hypothesis, 
and identification as confirmatory only those which did not 
include a biconditional interpretation, Moshman (1979, pg. 107) 
found that ••the mean numbers of implication interpretations (our 
of possible) for subjects in grades 7, 10 and college, 
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HYPOTHESIS TESTING 
respectively, were .75, 1.71, and 2.42." Lawson (1983) reported 
even more discouraging results. No subjects from a sample of 
students in an two introductory college biology courses chose 
only the falsifying instances in either of two hypothesis testing 
tasks. Thus, most studies continue to support the claim by 
dell'Aquila, Gennaro and Picciarelli (1985, pg. 71) that "few 
students show a real understanding of HT (hypothesis testing) 
logic.** 

Statement of the Problem 
Because of the probable link between an understanding of the 
nature of hypothesis testing and scientific reasoning, an attempt 
was made to teach the Bady **turtles** activity during an 
elementary science methods class for college students. The 
research questions were as follows: 

1) If pre-service teachers are taught to use falsification 
strategies in hypothesis testing, will transfer occur that leads 
to improved performance on a test of the ability to use 
prepositional logic; 

2) What relationship, if any, exists between the ability to 
use a hypothesis testing strategy and formal operational 
reasoning; and 

3) Are formal operational reasoning, the ability to use 
prepositional logic, and achievement in a science methods class 
related? 
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M«thod 

Subjects wer« 48 pr«-s«rvlc« students seeking certification 
and enrolled In an elementary science aethods course that met In 
a workshop format for one week. The core of the course was a 
group of 10 science centers. Additional activities Included 
films, lectures, readings, field trips and a variety of group 
activities. 

Propositions! logic was Introduced on the first day through 
a film about Plaget's classification schema, and Instruction was 
given on basic propositions and truth tables. Subsequently a 
group activity based on Bady's revision of the Hason and Johnson- 
Laird four-card task was conducted (Bady & Enyeart, 1978) . 

The ^'turtles'* activity asks students to consider the 
following statements, made by two biologists who have been 
studying a species of turtle that occurs only In a single remote 
lake. They are told that all of these turtles have either 
diamonds or circles on their back, and are either red cr green on 
their bottom. They are then asked how they might evaluate the 
following statements by these ba.^loglsts: 

BlQioaist ii All turtles with diamonds on their back have 

green bottoms. 

Bloloaiat #2 Biologist «1 Is wrong (It Is not true that all 

turtles with diamonds on their back have green 
bottoms) . 
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Th« activity occurs In thr«« phas«a. First, Individual inatancss 
(diamonds and grssn, diamonds and rsd, circles and green, circles 
and red) are presented, and a discussion held with the class. 
Next, students are presented with a "turtle pond** (overhead 
projection) that contains all turtles of this species, floating 
either on their backs or the bottoms, and asked which turtles 
they should turn over in order to settle the controversy. A 
lively discussion generally follows in which a numbed" o^ points 
of view are hotly contested (but unresolved by the Instructor) . 
Finally, the original Instances are presented again for analysis, 
and students fill out a form in which they identify the turtles 
that they would like to see in order to evaluate the hypothesis 
and state why they made that choice. 

In subsequent class meetings, two problems that are 
striicturally isomorphous with the "turtles" task (Mayer, 1987, 
pp. 34, 202) were Included on daily quizzes. These were: 
1) Suppose that someone made the following assertion, "If you 

get bitten by a black widow spider that has a red dot on its 

tummy, then you will get sick." Also suppose that you have 

four ways of testing this assertion: 

-You get bitten by a black widow spider that has a red dot, 
and check to see whether or rot you get sick. 
-You get bitten by a black widow spider that does not have a 
red dot, and check to see whether or not you get sick. 
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-You get sick fron a black widow spidar blta, and than chack 
to saa whathar or not tha spldar has a rad dot on Its bally. 
-You do not gat sick fro* a black widow spldar blta, and 
then check to see whether or not the spider has a red dot on 
its belly. 

2) If this is rooB 9, then it is the fourth grade. 

This is not room 9. Is it the fourth grade? 
Both items were graded, returned, and discussed the following day 
in class. 

From their responses to the second item analysis at the end 
of the "turtles'* activity, students were assigned to one of the 
following groups: 

III. FALSIFIERS. Correctly responded that only the diamonds 
and red instance was relevant, and that it falsified the 
hypothesis; 

II. TPANSITIONAL. Correctly responded to diamonds and red, 
but Incorrectly identified the diamonds and green instance as 
proving the hypothesis; and 

I. VERIFIERS. Incorrectly responded that they would 
search for diamonds and green turtles in an attempt to verify the 
hypothesis. 

The transfer, or 'target' task for this study was the 
Prepositional Logic Test (PLT) . This was given as a pre- and 
post-test (Plbum, 1989). This is a 16 item test of tha ability 
to identify Instances that are allowed or not allowed by 
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propositional statemtnts. It has a reportod reliability, for a 
sampla of 226 students, of .82. 

Tha Tast of Logical Thinking (TOLT) was given as a pre-test. 
This is a 10 item paper and pencil test of formal reasoning . 
Its reliability coefficient, based upon a sample of 982 students, 
is reported to be .85 (Tobin & Capie, 1981). 

Competency tests for centers required students to solve real 
problems similar to those with which they had been experimenting. 
For example, the test for a Batteries and Bulbs center required 
that they light a light bulb, state whether particular electrical 
circi^it diagrams would allow a light bulb to lighc, or draw the 
wires to complete a circuit. A Relative Position and Notion 
assessment provided carbon paper tracks of collisions between 
spheres, and required students to identify the conditions under 
which the tracks were made. 

Grades for the course were based upon a weighted average of 
performance on daily quizzes (10%), class participation (10%), 
completion of centers (25%), competency tests on centers (25%), 
and a student designed center (30%) . 

No estimates of reliability are available either for center 
assessments of course grade. 

Results 

The first research question addresses the degree to which 
instruction on a falsification strategy will transfer to 
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Increased success on a measure of the ability to reason about 
logical propositions. An Initial test of this transfer 
hypothesis was conducted by making a comparison of pre- and post- 
test scores on the Prepositional Logic Test. This analysis 
revealed a significant Increase In subjects* ability to Interpret 
logical statements (Table 1) . 



Insert table 1 about here 



In order to assess the degree to which transfer from trained 
to target task was evenly distributed across all students, 
subjects were grouped as falsifiers, transitional or verifiers on 
the basis of their performance In the "turtles" activity, and 
scores on the PLT were compared across these groups by means of 
one-way Analysis of variance (Table 2). There were significant 



Insert table 2 about here 



differences across the groups on both the pre-test and the post- 
test, with falsifiers performing better than verifiers In both 
comparisons. The scores of transitional subjects Increased until 
they were not significantly different from those of falsifiers on 
either the pre-test or the post -test. Although their scores 
Increased as well, verifiers continued to lag behind the other 
subjects on the post-test. 

13 

o U 

ERJ.C 



HYPOTHESIS TESTIK6 
The second research (juestion was whether formal thought was 
related to reasoning about logical propositions and hypothesis 
testing ability, correlations between score on the TOLT and the 
PLT were 0.29 for the pretest and 0.27 for the posttest, with the 
latter value narrowly failing to reach significance at a 
probability level of 0.05. When subjects were grouped according 
to their performance on the "turtles'* tasK, mean scores on the 
TOLT increased regularly from those of Verifiers (6.0) to 
Falsifiers (6.9) but the differences were not statistically 
significant. 

The final research question was about the relationships 
between the two measures of reasoning ability and achievement in 
the science methods class. A series of analyses were conducted 
in order to test the hypothesis that the PLT and the TOLT were 
correlates of success in the claus. The two achievement measures 
that were considered were performance on competency measures for 
centers and final grade in the class. A series of regression 
analyses were conducted in which pre-test scores on the PLT were 
regressed first against the dependent measure, and then the PLT 
posttest and TOLT scores were entered. This was done in order to 
eliminate initial differences that may have existed among members 
of the class. When scores on the PLT and TOLT variables were 
entered against competency measures for centers, the value of 
Beta for the PLT (0.27) was larger than that of the TOLT (0.16). 
On the other hand, when they were regressed against final grade 
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for the course, the value of Beta for the TOLT (0.29) was larger 
than that for the PLT (0.20). 

A principal components analysis was conducted in order to 
assess the relationship between intellectual and achievement 



insert table 3 about here 



variab3.es. This revealed a strong initial factor that contained 
all of the variables under consideration, as well as a second 
that contained only course success (Table 3) . 

Conclusion 

Even a brief intervention in logic can improve the quality 
of thinking of prospective teachers. A few hours during an 
elementary science methods course devoted to the nature of 
propositions, truth tables, and the falsification insight, led to 
significant increases in score on an unrelated test of the 
ability to use prepositional logic. Given the proven association 
between logic, formal thought and success in this methods class, 
the time spent on instruction in hypothesis testing was well 
invested. 

The development of the insight that hypotheses are not 
subject to verification, and the development of a falsification 
strategy were a crucial elements in this success. After 
instruction, from the entire group of 48 students, 16 (33%) fully 
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und«r8tood thesa concapts, 21 (44%) had partial mastary, and only 
*1 (23%) continuad to adopt a varification strategy. Tha first 
two groups achiavad a high rata o^ succass on tha Propositional 
Logic Test while the third continued to lag behind. A necessary 
step to improving performance by the latter group is a continued 
intervention on hypothesis testing strat^jgias. Only when these 
are correctly understood will all students achieve similar high 
rates of success in the interpretation of logical propositions. 

Although formal reasoning and reasoning about logical 
propositions were ahem to be related, there was no similar 
correspondence between the falsification insight and formal 
thought. These two measureiu are probably too different from one 
another for transfer to occur liiasily, and such a brief 
intervention is hardly likely to influence something as major as 
level of operational reasoning ability. But it is not 
unreasonable to believe that a continuing and more intensive 
course of instruction might have the desired result. 

Both formal operational reasoning and the ability to 
interpret logical propositions were significant elements in 
success in a science methods class. Reasoning about logical 
propositions might have been somewhat more important to success 
at center as:sassment8, whereas formal reasoning ability might 
have contributed more strongly to final grade in the course. 
That is not unreasonable, since course grade is a more global 
variable, as is formal thought. Center assessments were narrower 
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and mor« process cantsrsd, and ths ability to utlllza logical 
operators In problea solving could be expected to play a stronger 
part In their successful completion. 

Teachers who teach science must themselves be aware of the 
speculative nature of the enterprise. Rather than presenting 
science as a series of truths'* to be verified, they need to 
teach the logical constructs of hypothesis and falsification that 
constitute the normative procedures of theory testing In science. 
The evidence from this study Is that the structures of logic and 
the falsification Insight are closely related, and that Including 
appropriate Instruction In the pre-servlce training of elementary 
school teachers can be quite rewarding. 
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Table 1. Comparison of pr«- and post-test scores on 
the Proposltlonal Logic Test 
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PRETEST POSTTEST 
n 48 48 

mean 12.17 13.29 
S.D. 3.74 3.00 
t-valu« 2.89 
probability .0030 
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Comparison of pr«* and post-tsst maans on 
Propositional Logic Test by categories of 
perfoinaance on "turtles'* task 
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PRETEST POSTTEST 

Verifiers 9.75 11.63 

Intermediate 12.90 14.05 

Falsifiers 14.27 14.27 

F-ratlo 6.87 4.25 

significance .0028 .0199 
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Prlncipl* coapon«ntB analysis of three 
measures of reasoning ability, and two 
components of success in methods course. 
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Variabls 

Centsrs 
Coura* 

PraPLT 
PostPLT 

TOLT 

Eiganvalua 



Factor 1 

0.64 
0.72 
0.69 
0.74 
0.59 

2.30 



Factor 2 

0.62 

0.56 
-0.60 
-0.50 
-0.02 

1.30 
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